__author__ = 'Susan Ridgway <susan.ridgway@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
__version__ = '20230821' # yyyymmdd
__datasets__ = ['GS-2018A-Q-207'] # Data comes from the Gemini archive, GS-2018A-Q-207
__keywords__ = ['DRAGONS', 'Gemini', 'GMOS', 'data reduction']
Authors: Susan Ridgway, Vinicius Placco
Adapted from https://dragons.readthedocs.io/projects/gmosimg-drtutorial/en/v3.1.0/
Showcase how to perform GMOS imaging data reduction using the Gemini DRAGONS package on the Data Lab science platform. Uses a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing of flats, bias, and science frames, and finally the creation of a single combined stacked image.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It currently can be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, and spectroscopic data in GMOS longslit mode. Linked here is a general list of guides, manuals and tutorials about the use of DRAGONS: https://dragons.readthedocs.io/en/v3.1.0/
The DRAGONS kernel has been made available in the Data Lab environment, which should allow users to access the routines without being dependent on installing the software in their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to fully reduce example Gemini South GMOS G band imaging data. This is a version of the DRAGONS Jupyter notebook tutorial presented here: https://gitlab.com/nsf-noirlab/csdc/usngo/DRAGONS_tutorials/-/blob/main/GMOS_IM_FIELD.ipynb
This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GMOS data reduction process, rather will just show one example of a standard reduction of a GMOS imaging dataset. More extensive explanations can be found in the general DRAGONS GMOS data reduction tutorial from Gemini linked here: https://dragons.readthedocs.io/projects/gmosimg-drtutorial/en/v3.1.0/
The data used in this notebook example is GMOS G band imaging from the Gemini archive of the galaxy NGC 5018 from the Gemini South program "The Evolutionary History of NGC 5018", PI: L. Sesto, program ID GS-2018A-Q-207. More program information is given here: https://archive.gemini.edu/programinfo/GS-2018A-Q-207. The final reduced science image combines 5 science frames of 460 seconds each, that were dithered between each exposure.
If you use this notebook for your published science, please acknowledge the following:
Data Lab concept paper: Fitzpatrick et al., "The NOAO Data Laboratory: a conceptual overview", SPIE, 9149, 2014, http://dx.doi.org/10.1117/12.2057445
Data Lab disclaimer: https://datalab.noirlab.edu/disclaimers.php
DRAGONS publication: Labrie et al., "DRAGONS - Data Reduction for Astronomy from Gemini Observatory North and South", ASPC, 523, 321L, https://ui.adsabs.harvard.edu/abs/2019ASPC..523..321L/abstract
DRAGONS open source software publication: https://zenodo.org/record/7776065#.ZDg5qOzMLUI
# std lib
import glob
# 3rd party
from astropy.io import fits
from astropy.wcs import WCS
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
%matplotlib inline
# DRAGONS
from recipe_system import cal_service
from recipe_system.reduction.coreReduce import Reduce
from gempy.adlibrary import dataselect
from gempy.utils import logutils
# Setup a logfile to keep track of the DRAGONS reduction
logutils.config(file_name='reduce.log')
Create a list of the file names of the raw GMOS G band imaging data, and use wget to download these from the Gemini archive.
%%bash
# create file that lists FITS files to be downloaded
echo "\
http://archive.gemini.edu/file/S20180419S0098.fits
http://archive.gemini.edu/file/S20180419S0099.fits
http://archive.gemini.edu/file/S20180419S0100.fits
http://archive.gemini.edu/file/S20180419S0101.fits
http://archive.gemini.edu/file/S20180419S0102.fits
http://archive.gemini.edu/file/S20180419S0207.fits
http://archive.gemini.edu/file/S20180419S0208.fits
http://archive.gemini.edu/file/S20180419S0209.fits
http://archive.gemini.edu/file/S20180419S0210.fits
http://archive.gemini.edu/file/S20180419S0211.fits
http://archive.gemini.edu/file/S20180423S0050.fits
http://archive.gemini.edu/file/S20180423S0051.fits
http://archive.gemini.edu/file/S20180423S0052.fits
http://archive.gemini.edu/file/S20180423S0053.fits
http://archive.gemini.edu/file/S20180423S0054.fits\
" > gmos.list
# download with wget FITS files from Gemini archive for processing (in quiet mode)
wget --no-check-certificate -N -q -i gmos.list
Starts the calibration service for DRAGONS. The w flag removes all previous calibration databases that might have been set up.
caldb = cal_service.set_local_database()
caldb.init("w")
This cell stores the names of the files and prints a sorted list for visual inspection.
all_files = glob.glob('S20*[0-9].fits')
all_files.sort()
print(all_files)
['S20180419S0098.fits', 'S20180419S0099.fits', 'S20180419S0100.fits', 'S20180419S0101.fits', 'S20180419S0102.fits', 'S20180419S0207.fits', 'S20180419S0208.fits', 'S20180419S0209.fits', 'S20180419S0210.fits', 'S20180419S0211.fits', 'S20180423S0050.fits', 'S20180423S0051.fits', 'S20180423S0052.fits', 'S20180423S0053.fits', 'S20180423S0054.fits']
Here the raw data is sorted into types (bias, flats, and science images) using dataselect and lists are made of the raw frames of each type. These lists are used as input to the later combination steps.
list_biases = dataselect.select_data(all_files,['BIAS'],[])
list_flats = dataselect.select_data(all_files,['FLAT'],[],
dataselect.expr_parser('filter_name=="g"'))
list_science = dataselect.select_data(all_files,[],['CAL'],
dataselect.expr_parser('(observation_class=="science" and filter_name=="g")'))
The bias frames are reduced and stacked to create a combined bias frame. The flat frames are reduced, bias corrected and stacked to create a combined flat frame. After each reduction, the output is added to the calibration database.
# This should take about 45 seconds
reduce_bias = Reduce()
reduce_bias.files.extend(list_biases)
reduce_bias.runr()
caldb.add_cal(reduce_bias.output_filenames[0])
reduce_flats = Reduce()
reduce_flats.files.extend(list_flats)
reduce_flats.runr()
caldb.add_cal(reduce_flats.output_filenames[0])
All submitted files appear valid: S20180423S0050.fits ... S20180423S0054.fits, 5 files submitted.
Ginga not installed, use other viewer, or no viewer
In this step, the standard DRAGONS GMOS data reduction protocol is followed: first, the science frames are processed, and a rough combined frame is created that is then used to further flat field and cosmic ray reject the input science frames. These further processed, dithered input frames are then re-combined to maximize signal-to-noise and produce a final combined science frame. Note: This process is iterative: 1st, detects sources in individual frames, cross correlated to see which are cross correlated (not CRs) and also uses these to derive offsets, and CR correct. A combined (median, no rejection) sky frame is made, which is then subtracted from each image the final science frame is created after CR Rejecting and sky subtracting the input files, using combination = mean and rejection sigclip
# This should take about 2 minutes
reduce_science = Reduce()
reduce_science.files.extend(list_science)
reduce_science.runr()
Read the reduced spectrum with astropy and print information about the various FITS extensions provided by the DRAGONS data reduction. Also read the WCS (World Coordinate System) for plotting purposes.
image_file = "S20180419S0098_image.fits"
hdu_list = fits.open(image_file)
wcs = WCS(hdu_list[1].header)
hdu_list.info()
WARNING: FITSFixedWarning: 'datfix' made the change 'Set DATE-OBS to '2018-04-19T03:27:58.541' from MJD-OBS'. [astropy.wcs.wcs] FITSFixedWarning: 'datfix' made the change 'Set DATE-OBS to '2018-04-19T03:27:58.541' from MJD-OBS'.
Filename: S20180419S0098_image.fits No. Name Ver Type Cards Dimensions Format 0 PRIMARY 1 PrimaryHDU 250 () 1 SCI 1 ImageHDU 148 (3245, 2182) float32 2 VAR 1 ImageHDU 148 (3245, 2182) float32 3 DQ 1 ImageHDU 148 (3245, 2182) int16 (rescales to uint16) 4 PROVENANCE 1 BinTableHDU 17 7R x 4C [28A, 128A, 128A, 128A] 5 PROVHISTORY 1 BinTableHDU 17 18R x 4C [128A, 288A, 28A, 28A]
Get (x,y) data from the SCI extension and print the array's shape
image_data = fits.getdata(image_file, ext=1)
print(image_data.shape)
(2182, 3245)
Display the image using standard matplotlib.pyplot routines. Note the use of the WCS for the coordinates. The norm=LogNorm function is used to improve the contrast in the image.
plt.figure(figsize = (15,15))
plt.subplot(projection=wcs)
plt.imshow(image_data,cmap='gray',norm=LogNorm(vmin=3000, vmax=60000))
plt.xlabel('Right Ascension [degree]',fontsize=14,fontweight='bold')
plt.ylabel('Declination [degree]',fontsize=14,fontweight='bold')
plt.xlim(900,2500)
plt.ylim(1600,600)
plt.show()
Erase all fits files, lists, logs, and calibrations created during the reduction. For that, uncomment all lines in the cell below and execute.
#%%bash
#
#rm -f *S2018*.fits gmos.list reduce.log # remove files
#rm -rf calibrations/ # remove directories
US NGO DRAGONS tutorials: https://gitlab.com/nsf-noirlab/csdc/usngo/DRAGONS_tutorials/
US NGO DRAGONS tutorial on which this example is based: https://gitlab.com/nsf-noirlab/csdc/usngo/DRAGONS_tutorials/-/blob/main/GMOS_IM_FIELD.ipynb
Gemini DRAGONS list of manuals and tutorials: https://dragons.readthedocs.io/en/v3.1.0/
Gemini DRAGONS GMOS imaging data reduction tutorial:
https://dragons.readthedocs.io/projects/gmosimg-drtutorial/en/v3.1.0/
https://archive.gemini.edu/programinfo/GS-2018A-Q-207